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VISUAL IMAGE SENSOR ORGAN 
REPLACEMENT: IMPLEMENTATION 

RELATED APPLICATION 

This application is a continuation-in-part of a patent appli- 
cation entitled “Visual Image Sensor Organ Replacement” 
U.S. Ser. No. 1 1/239,450, filed 23 Sep. 2005. 

ORIGIN OF THE INVENTION 

This invention was made, in part, by one or more employ- 
ees of the U.S. government. The U.S. government has the 
right to make, use and/or sell the invention described herein 
without payment of compensation, including but not limited 
to payment of royalties. 

FIELD OF THE INVENTION 

This invention relates to implementation of use of audio 
signal parameters as representatives of time varying or con- 
stant visual signal parameter values. 

BACKGROUND OF THE INVENTION 

Present development of fast, cheap and miniaturized elec- 
tronics and sensory devices opens new pathways for the 
development of sophisticated equipment to overcome limita- 
tions of the human senses. Humans rely heavily on vision to 
sense the environment in order to achieve a wide variety of 
goals. However, visual sensing is generally available only for 
a limited visible range of wavelengths, roughly 400 nm (na- 
nometers) to 720 nm, which is a small fraction of the range of 
wavelengths (180 nm through about 10,000 nm) at which 
interesting physical effects and/or chemical effects occur. 
Audible sensing, over an estimated audible range of 200 Hz 
(Hertz)-20,000 Hz, is similarly limited, but this range is a 
larger fraction of the audibly interesting range 1 0 Hz- 1 0 5 Hz). 
Further, use of binaural hearing to provide audible clues as to 
depth and relative location is generally better developed than 
are the corresponding mechanisms associated with formation 
of visible images. 

Since the time of Aristotle (384-322 BC), humans have 
been interested in perceiving what is beyond normal “vision”. 
Roentgen’s discovery of X-Rays enabled him to see inside 
living tissue, and “vision” was thereby extended beyond the 
naked eye. In the following years, imaging and sensing tech- 
niques have developed so rapidly that astronomy, medicine 
and geology are just few of the areas where sensing beyond 
the normal visual spectrum has been found useful. Altering 
and extending human “vision” changes our perception of the 
world 

According to some recent research in evolution of the sight 
system for animals, reported in “What Birds See” by Timothy 
H. Goldsmith, Scientific American, July 2006, pp. 68-75, 
certain bird species have a tetra-chromatic color sensing sys- 
tem, with color bands spanning the near-ultraviolet, violet, 
green and red wavelengths, in contrast to the tri-chromatic 
(for primates, humns and some birds) or bi-chromatic (for 
other animals) color sensing systems that cover only two or 
three visible wavelength bands. The tetra-chromatic color 
sensing system of the birds allows more subtle sensing of 
color differences, much as HD radio claims to allow receipt of 
radio frequencies between the 0.2 kHz signposts of conven- 
tional commercial radio. This extra color sensing subtlety 
available to some birds is not available, and is not likely to 
become available, generally to humans and/or primates. 
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Further, the human audible sensing system is capable of 
learning to process and interpret extremely complicated and 
rapidly changing auditory patterns, such as speech or music in 
a noisy environment. The available effective bandwidth, on 
5 the order of 20 kHz, may support a channel capacity of several 
thousand bits per second. The known capabilities of the 
human hearing system to learn and understand complicated 
auditory patterns provide a basic motivation for developing a 
visual image-to -sound mapping system, 
to What is needed is a system that converts “visual signals”, 
defined herein as signals with at least one associated wave- 
length in the ultraviolet, the visible and/or the infrared, to one 
or more audibly perceptible signals with associated audio 
parameters that can be recognized and distinguished by the 
15 human ear. Preferably, these signals should include an 
audible indication of change, or change rate with time, of one 
or more visual image parameters. Preferably, these audio 
signals should provide monaural and/or binaural signaling 
that is analogous to depth clues and/or distance clues pro- 
20 vided by visually perceptible images. Preferably, the audible 
signal parameters should have an intuitive connection with 
the visual signal parameters to which the audible signal 
parameters correspond. 

25 SUMMARY OF THE INVENTION 

These needs are met by the invention, which provides a 
mapping or association between signals representing a 
selected region of a received visual image and audibly per- 
30 ceptible signals in which M visual signal parameter values 
(M=l-8) are mapped one-to-one onto a selected set of distin- 
guishable audible signal parameters. External multi-spectral 
sensors signals are translated into audible signals targeting 
the same human visual field. 

35 The visual signal is received and one or more visual signal 
parameters are measured or otherwise provided, including 
but not limited to distinction between visual signal wave- 
lengths in the ultraviolet, the visible, the near-infrared and the 
mid-infrared. The audible signal parameter values provided 
40 as output include one or more of: an envelope frequency f e ; a 
time rate of change of the envelope frequency (analogous to 
“chirping” or to a Doppler effect); a carrier frequency f c ; an 
envelope frequency phase (|) e at a selected time, t=t^ e ; a 
carrier frequency phase f c at the selected time, t=t^ c ; a base- 
45 line function b(t) the defines a baseline curve BB; a time rate 
of change db/dt of the baseline function; a non-undulatory, 
but possibly time varying, signal amplitude a(t), measured 
relative to the baseline curve BB; and a time interval (dura- 
tion) At for the signal. The human ear may be able to distin- 
50 guish the phase difference, A(|)=(|) e -(|) c , but need not recognize 
the absolute phases, § e and/or § e . 

BRIEF DESCRIPTION OF THE DRAWINGS 

55 FIG. 1 graphically illustrates several audio signal param- 
eters that can be used in the invention to provide a visual - 
audio association of parameters. 

FIG. 2 schematically illustrates (partial) representation of a 
6Q sequence of regions of a visual image. 

FIG. 3 schematically illustrates a mapping device used to 
transform visual image region parameters to audibly percep- 
tible signal parameters. 

FIG. 4 illustrates a suitable receiver/processor used in FIG. 

65 3- 

FIG. 5 is a flow chart illustrating practice of an embodiment 
of the invention. 
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FIG. 6 illustrates an application of the system in a battle- 
field situation. 

FIG. 7 graphically illustrates variation of estimated projec- 
tile impact effective distance d(E), for death or serious injury 
to a nearby combatant or disablement of an equipment item, 5 
as the projectile explosive load E varies. 

FIGS. 8 A, 8B, 9A and9B graphically illustrate frequencies 
and intensities of audible signals used in different versions of 
a projectile impact example. 

FIG. 10 schematically illustrates determination of present to 
location of a projectile, using observations from two spaced 
apart observation sites. 

FIG. 11 schematically illustrates distinction between tra- 
jectories of two different projectiles. 

15 

DESCRIPTION OF BEST MODES OF THE 
INVENTION 

FIG. 1 graphically illustrates M signal parameter values 
(M= 1-8) that can be used to collectively characterize an undu- 20 
lating, audibly perceptible signal s a (t) having a single infor- 
mation-bearing (envelope) frequency and a single carrier fre- 
quency. This signal can be characterized by: an envelope 
frequency f e and corresponding time rate of change of the 
envelope frequency df e /dt (analogous to “chirping” or to a 25 
Doppler shift); a carrier frequency f c ; an envelope frequency 
phase § e at a selected time, t=t p eh ; a carrier frequency phase 
f c at the selected time, t=t ph c ; a baseline function amplitude 
b(t), defining a baseline curve BB, and corresponding time 
rate of change of base line amplitude db / dt; a non-undulatory 30 
signal amplitude a(t), measured relative to the baseline curve 
BB; and a time interval (duration) At for the signal. The 
human ear may be able to distinguish the phase difference, 
A(|)=(|) e -(|) c , but cannot distinguish the absolute phases, § e 
and/or <j) e . An audible signal equation incorporating all these 35 
features is 

SJt)=b(t)+a(t)- sin {f c t+$ c } (1) 

The maximum number of parameters for the signal shown 
in FIG. 1 that may be distinguished by the human ear is 40 
M=6-8, if the (absolute) selected time, t=t^, and the absolute 
phases are not included. These M signal parameters may be 
used to audibly represent a corresponding visual region of an 
image, such as vertical and horizontal coordinate ranges (ver- 
sus time) of the visual region (relative to a fixed two-dimen- 45 
sional or three-dimensional system), estimated distance s(t) 
and/or rate of change of distance ds/dt to a selected center of 
the region, region brightness, overall region brightness, and 
region predominant hue (color) or wavelength. Optionally, 
these audible signal parameters can be presented simulta- 50 
neously or sequentially, for any corresponding visual image 
region that is so represented. In a sequential presentation, one 
or more additional audible signal parameters may be 
included, if the information corresponding to the additional 
parameter value(s) is necessary for adequate representation 55 
of the image region. 

The visual image may be decomposed into a sequence of K 
selected visual image regions R*, (k=l, . . . , K, with K^l; 
contiguous or non-contiguous; overlapping or nonoverlap- 
ping) that make up part or all of the total visual image, for 60 
example as illustrated in FIG. 2. The sequence of regions R^, 
and the corresponding sequence of audible signal parameters, 
need not exhaust the set of all regions that together make up 
the visual image. Preferably, the image regions are chosen 
according to which regions are of most interest. For example, 65 
when an image has a single image region (less than the entire 
image) where one or more image parameters is changing 


4 

substantially with time, this region may be a primary focus; 
and if this region slowly changes its location or its physical 
extent within the total image, the location and breadth of this 
image region should correspondingly change with time. That 
is, the horizontal and vertical bounds and/or the center of an 
image region may move with time within the total image. 

If the visual image changes between one time to a subse- 
quent time, the audible parameters representing each selected 
region R k may also change with time, in a sequential manner. 
FIG. 1 graphically illustrates signal parameters correspond- 
ing to a mapping that can be implemented to represent a group 
of visual signal parameters, representing a selected region R /f 
of the total image, by an audibly perceptible signal or signals. 

In one approach, a visual image region R^ is selected and 
optionally isolated, and the corresponding audibly percep- 
tible signal parameters are presented (1 ) sequentially within a 
time interval of selected length (e.g., 5-30 sec) or (2) as part of 
a single audible signal that incorporates two or more selected 
audible signal parameter values. 

If an audible signal parameter changes with time, continu- 
ously or discretely, this change can be presented according to 
several options: (i) change the audible parameter value con- 
tinuously at a rate that corresponds to the time rate of change 
of the corresponding visual parameter value; (ii) change the 
audible parameter value discretely at a rate corresponding to 
a discrete time rate of change of the visual parameter value; 
and (iii) change the audible parameter value discretely, by a 
selected amount, only when the magnitude of the difference 
between a first value and a second value of the parameter is at 
least equal to a threshold magnitude, which will vary with the 
nature of the visual parameter. 

Humans and primates rely heavily on tri -chromatic vision 
to sense and react to the environment in order to achieve 
various goals. By contrast, other animals rely heavily, but not 
exclusively, on smell (e.g., rodents), on sound (e.g., some 
birds), or on tetra-chromatic vision (other birds). The inven- 
tion augments or replaces a human sensory visual system, 
which is deficient in many respects, with one or more auditory 
signals, in order to achieve the following. 

(1) Provide a capacity to sense beyond the human visible 
light range of the electromagnetic spectrum). 

(2) Increase capacity of human sensing resolution, beyond 
the number of rods and cones in the human eye (approxi- 
mately 120 million rods and 6 million color sensing cones) 
that limit the resolution of the images, particularly because 
humans rely on the subset of cones located in the fovea, which 
provide humans the highest visual acuity of approximately 1 
min of arc resolution within a field of view less than 12 
degrees horizontal by 4 degrees vertical in humans. 

(3) Provide wider angle equivalent of visual sensory per- 
ception, where the shape and location of human eyes limit the 
effective human field of view to about 200 degrees horizon- 
tally by about 150 degrees vertically. 

(4) Improve the ability of a human to sense distances, 
which is presently relatively poor and can be confounded by 
a wide variety of visual cues. 

(5) Allow compensation for movement by the human or 
changes in the scene; for example, motion smear or blur can 
make it difficult to resolve images at resolutions achievable 
when the perspective of an image is not moving or changing. 

(6) Allow splitting of user attention (multi-tasking using 
two or more senses), where a visual image limits the range of 
other activities that a person can do simultaneously, such as 
monitoring gauges and reading text concurrently. 

(7) Provide audibly perceptible changes in an audible 
parameter value that correspond to changes, continuous or 
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discrete, in a visual parameter value that are too small or 
subtle for a human eye to sense or respond to. 

(8) Provide an audible parameter value that changes in an 
audibly perceptible manner only when the corresponding 
visual parameter changes by at least a threshold amount, and 
the threshold is selectable according to the environment. 

Using the invention, a wide variety of tasks that are difficult 
or cumbersome to accomplish, using primarily visual indicia, 
can be met, including the following: 

Enabling the user to substantially simultaneously focus 
attention on multiple aspects of a visual field, or its 
audible field equivalent; 

Embedded human sensing of aircraft performance; and 

Audibly indicating micro-fractures and/or thermal distor- 
tion in materials. 

In order to increase the visual image resolution obtainable 
via an auditory representation, a mapping is performed to 
distribute an image in time. Three-dimensional spatial bright- 
ness and multi-spectral maps of a sensed image are processed 
using real-time image processing techniques (e.g., histogram 
normalization) and are transformed into one or more two- 
dimensional maps of an audio signal as a function of fre- 
quency and of time. 

The invention uses a Visual Instrument Sensory Organ 
Replacement (VISOR) system to augment the human visual 
system by exploiting the improved capabilities of the human 
auditory system. The human brain is far superior to most 
existing computer systems in rapidly extracting relevant 
information from blurred, noisy, and redundant images. This 
suggests that the available auditory bandwidth is not yet 
exploited in an optimal way. Although image processing tech- 
niques can manipulate, condense and focus the information 
(e.g., using Fourier Transforms), keeping the mapping as 
direct and simple as possible may also reduce the risk of 
accidentally filtering out important clues. Even a perfect, 
non-redundant sound representation is subject to loss of rel- 
evant information in a non-perfect human hearing system. 
Also, a complicated, non-redundant visual image-to -audible 
image mapping may well be more difficult to learn and com- 
prehend than a straightforward visual mapping, while the 
mapping system would increase in complexity and cost. 

FIG. 3 schematically illustrates a mapping device used to 
transform selected visual image parameters associated with a 
region to an audible signal with audibly perceptible param- 
eters. One or more visual image region (“VIR”), representa- 
tions is received and analyzed by a first stage signal receiver- 
processor (“R/P”) 31-1. The first stage R/P 31-1 analyzes a 
received VIR and provides one or more (preferably as many 
as possible) of the following visual signal characterization 
parameters in a second stage R/P 31-2: vertical and horizontal 
coordinate ranges of the region and/or its center; optional 
adjustment in size of region viewed; region predominant hue 
(color) or wavelength; region average brightness and region 
peak brightness, using a region locator and sizing mechanism 
32, a region predominant (or average) color sensing mecha- 
nism 33 and a region brightness sensing mechanism 34. Out- 
put signals from the locator mechanism 32, from the color 
mechanism 33 and from the brightness mechanism 34 are 
received by a third stage R/P 31-3, which provides a collec- 
tion of audible signal parameters, including time rate of 
change TRC of at least one parameter value. 

As an example: the predominant or average color output 
signal from the region color sensing mechanism 33 can be 
used to determine the envelope frequency f e ; the region 
brightness output signal from the region brightness mecha- 
nism 34 can be used to determine the envelope relative ampli- 
tude, a 0 (constant) or a(t); the vertical and horizontal location 
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output signals from the locator mechanism 32 can be used to 
determine time duration At (if the visual image region loca- 
tions are indexed by a one-dimensional index), or to deter- 
mine time duration At, envelope frequency f e (if the visual 
5 image region locations are indexed using a two-dimensional 
index) or change rate, db/dt or df e /dt. The four visual signal 
parameters can be assigned to four of six audibly perceptible 
signal parameters (FIG. 1) in ( 6 4 )=(6*5*4-3)/(4*3*2T)=15 dis- 
tinguishable ways. More generally, N visual signal param- 
10 eters can be assigned to M (=N) audibly perceptible signal 
parameters in (^^y) distinguishable ways. 

Where the time rate of change option (i) is used for a visual 
signal parameter value r, one can form an approximating 
second degree polynomial 
15 

r(r;app)={r(g{(r-^ + i)(r-^ +2 )(^ + 2 -r i?+ i)+r(^ + i){(r- 

g^-g 2 )(g2-g+Kg2){h-gh-gi)(gi- 



( 2 ) 


( 3 ) 


(t i < t 2 < . . . < tp < tp + 1 < tp + 2 < • • • ) and compute r(t) and dr/dt 
using the approximating polynomial r(t; app) and dr(t; app)/ 
dt, respectively. Approximating polynomials of degree higher 
than two can also be used here. 

Where the time rate of change option (ii) is used for a visual 
signal parameter value r, a sequence of ratios 

v 2{t p ;t p+ ^={r{t p+i )-r(t^)}l(t p+i -t^), (4) 

is computed for the sequence of times {g}. 

Where the time rate of change option (iii) is used for a 
visual signal parameter value r, the time rate of change ratios 
of interest become 


v3 {t p \ tp+p ) = 0 (if| r(t p ) - r(t p+q )\ < A r(thr) for q = 1 , 2, . . . , P - p) (t 
= Htp+p ) - r(tp)}/(T p+P - r p )(if|r(r p ) - r{t p+q )\ > A r(thr) 
for q = 1, 2, . . . , P - p - 1 and | r(t p ) - r(t p+ p)\ > A r(thr). 

40 

The analysis performed by each of the mechanisms 32, 33 
and 34 is not instantaneous, and the associated time delays 
may not be the same for each analyzer. For this reason, an 
45 overall time delay 

Ar(o 1 ) ^min{ Allocation, size) ,A?(color) ^(bright- 
ness)} (6) 

is preferably imposed, using a time delay mechanism 35, 
50 before an audible signal (or audible signal sequence) incor- 
porating the 1 through M=M1+M2 audible signal parameters 
is audibly displayed, where M2 is the number of parameter 
values that can change with time and Ml is the number of 
remaining parameters. If the audible signal parameters are 
55 displayed sequentially, mot simultaneously or collectively, 
this time delay might be reduced or eliminated. The overall 
time delay is implemented by a time delay mechanism 35, 
which incorporates an appropriate time delay value for each 
of the audible signal parameters received from the first stage 
60 R/P 31-1. An audible signal formation mechanism 36 (op- 
tional) forms and issues either: (1) an audibly perceptible, 
ordered sequence of the set of M audible signal components 
ASC(m), m=l, . . . , M, (or a subset thereof), or (2) a collective 
audibly perceptible signal APS incorporating the set (or a 
65 subset) of the audible signal components. The output signal 
from the audible signal formation mechanism 36 is perceived 
by a human or other animal recipient. 
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The R/P 40, illustrated in FIG. 4, includes one or more of 
the following; a carrier/envelope frequency (f c ) analyzer 41; 
an envelope amplitude analyzer 42; an envelope-carrier fre- 
quency phase difference (A(|)) analyzer 43 and baseline func- 
tion (b(t)) analyzer 44 that estimate the phase difference at a 5 
selected time and determines the baseline function and base- 
line time rate of change; and a relative signal amplitude (a 0 or 
a(t)) analyzer 45, relative to the baseline function at a corre- 
sponding time. 

The analysis performed by each of these analyzers is not 10 
instantaneous, and the associated time delays may not be the 
same for each analyzer. For this reason, an overall time delay 

Ar(o2)^min{ ht(f e ) ,A t(f c ) ,Ar(A<|>) ,Af(6),Af(a)} (7) 

is preferably imposed before an audible signal incorporating 15 
the M converted visual signal parameters is audibly dis- 
played. If the converted visual signal parameters are audibly 
displayed sequentially, rather than simultaneously, this time 
delay might be reduced or eliminated. The overall time delay 
is implemented by a time delay mechanism 46, which incor- 20 
porates an appropriate time delay for each of the audible 
parameters received from the R/P 31. A signal formation 
module 47 forms a composite audible signal representing an 
audible image component, and issues this component as an 
output signal. 

Where the amplitude a(f) is constant, the signal shown in 
FIG. 1 may be represented in an alternative form 

Fvir = b{t) + aosm{f e (t - r^) + A0)}sin{/ e (r - fy)} (8) 30 

= b{t) + a 0 {cos{(/ c - f e )(t - fy) - A <p) - a 0 {cos{(/ c + f e ) 

h- 1 #) + a$>} 

35 

The carrier/envelope frequency analyzer 42 forms a 
sequence of correlation signals, computed over a time interval 
of length T, 



(9A) 

C2(fJ=(l/T)fF„ R (t)c OS {f cs t)dt. 

(9B) 


at each of a spaced apart sequence of “translated” carrier 
frequencies, f cl in a selected carrier frequency range, 45 
f cl =f cs =f c2 , where f cs is not yet known, and provides an 
estimate of two spaced apart frequencies Ui=f>f e and 
f cs2 =f c -f e , associated with the VIR, where the correlation 
combination, Cl 2 +C2 2 , has the highest magnitudes. The 
envelope and carrier frequencies are then estimated from 5Q 

fc=(fcsi+Ls2y2, (10A) 

fe=(fcsl-fcs2)l2. (10B) 

The envelope-carrier phase difference A(|) and relative 
amplitude a 0 are determined by computing the correlations 

{l/T)$Fyj R {t)sm {f e (t-t^j}dt=a 0 cos A<|), (1 1A) 

{l/T)$Fyj R {i)cos { f e (t-t^)}dt=a 0 sin A<1>, (11B) 

60 

from which the quantities a 0 (i^O) and A(|) are easily deter- 
mined. The baseline function b(t) is then determined from 

b{t)=F ^(r)-a 0 {cos {(/;-/J(r-4)-A(l)}-flo{cos {{f c + 

f e )(t-t+)+ A<|>}. (12) 

65 

The frequency difference (f c -f e ) and frequency sum (f c +f e ) 
values are distinguished from each other in a normally func- 
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tioning human auditory system if the sum-frequency differ- 
ence is at least equal to a threshold value, such as 250 Hz. 

FIG. 5 is a flow chart illustrating a method for practicing 
the invention. In step 51, at least one selected region of a 
visual image is represented by N selected visual image 
parameters, including at least one of: vertical and horizontal 
location coordinates and time rate of change of location coor- 
dinate^), relative to a signal recipient, of the region; region 
predominant hue or wavelength, region brightness (average 
and/or peak); and time rate of change of a visual signal param- 
eter. In step 52, the visual image region representatives are 
mapped onto M audible signal attributes (Mi^N), drawn from 
the following set of attributes: carrier signal frequency; enve- 
lope signal frequency and time rate of change of envelope 
frequency; carrier signal-envelope signal phase difference at 
a selected time; baseline amplitude and time rate of change of 
baseline amplitude; envelope signal amplitude relative to 
baseline amplitude; and signal time duration. In step 53, the 
audible signal attributes are presented sequentially in an audi- 
bly perceptible manner to the recipient. In an alternative step 
53 (step 54), the audible signal attributes are received and 
incorporated in one or more audible signals that is/are pre- 
sented in an audibly perceptible maimer to a recipient. 

The invention can be applied to provide audibly percep- 
tible and distinguishable signals, representing one or more 
selected regions of a visually perceptible image, for a sight- 
impaired person. Where more than one VIR is represented, 
the audible signal representatives of the VIRs are preferably 
presented sequentially, with a small separation time interval 
(as little as a few tens of msec) between consecutive repre- 
sentatives. 

Where the visual image is a line drawing or other binary 
representation, the audible signal components can be config- 
ured to represent curvilinear and linear shapes, sizes and 
intersections. Where the visual image primarily represents 
interaction of dominant color masses in different regions of 
the image, the dominant hues and shapes of these interacting 
regions can be represented audibly. For other reasons, a non- 
sight-impaired person may prefer to focus attention on 
attributes of a region of an image that can be represented more 
accurately or intuitively by non-visual signals, for example, 
to extend the (visual) wavelength range of signals that can be 
perceived. 

The invention can be applied to “enrich” image detail or 
manifest more clearly some image details that are not evident 
where the region is viewed solely with reference to visible 
light wavelengths. For example, some details of a region may 
be hidden or muddled when viewed in visible wavelength 
light but may become clear when the region is illuminated 
with, or viewed by, an instrument that is sensitive to, near- 
infrared light (wavelength X«0.7-2 pm) or mid-infrared light 
(X«l-20 pm) or ultraviolet light (X^0.4 pm). These hidden 
details can be converted to audible signal parameter values 
that are more easily audibly perceived as part of a received 
signal. Operated in this manner, the invention can separately 
compensate for a relatively narrow (or relatively broad) vis- 
ible wavelength sensitivity of the viewer and a relatively 
narrow (or relatively broad) auditory frequency sensitivity of 
the same viewer or of a different viewer-recipient. Operated 
in this manner, the visible wavelength sensitivity of a first 
(visual image) viewer of the image region can be adjusted and 
compensated for electronically by adjusting the audible 
wavelength range of one or more of the audible signal param- 
eters, before the transformed audible signal is received by the 
same viewer or by a different viewer. 

The invention can also be applied to provide audible signal 
components representing shape signatures, sizes and esti- 
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mated separation distances for objects that cannot be seen, or 
that are seen very imperfectly, because of signal interference, 
signal distortion and/or signal attenuation by the ambient 
environment. This may occur in a hazardous environment 
where fluids present provide an opaque, darkened or translu- 5 
cent view of objects in the environment, including moving or 
motionless persons and objects that present a hazard 

This interference may also occur in an airborne environ- 
ment in which rain, snow, hail, sleet, fog, condensation and/or 
other environmental attributes prevent reasonably accurate to 
visual perception of middle distance and far distance objects. 

A visual image region that is likely to experience interference 
can be converted and presented as a sequence of audio signal 
attributes that can be more easily or more accurately per- 
ceived or interpreted by an operator of an aircraft (airborne or 15 
on the ground). The audio signal attributes may be extended 
to include estimated closing velocity between the operator/ 
aircraft and the not -yet-seen object. 

The invention can also be applied, in an environment of 
visual “confusion,” to focus on and provide information only 20 
on important details among a clutter of unimportant details. 
The important details may be characterized by certain param- 
eters, and the system may focus on initially- visual details that 
possess one or more of these parameters, converting the rel- 
evant information to audibly perceptible signals that contain 25 
this (converted) information. An example is a specified air- 
craft approaching a destination airport surrounded by other 
airborne aircraft: the specified aircraft may wish to focus on 
and receive relevant (converted) audibly perceptible informa- 
tion for the immediately preceding aircraft and the immedi- 30 
ately following aircraft in a queue formed by an air traffic 
controller to provide an orderly sequence of touchdowns at 
the destination airport. 

The invention can also be applied where visual signals 
representing the image are more likely to experience signal 35 
interference, signal distortion, signal attenuation and/or simi- 
lar signal impairments than are selected corresponding 
audible signals that represent certain parameters in these 
visual signals. The visual signals (now converted to audible 
signals) may be transmitted through the ambient environment 40 
with reduced signal interference, reduced signal distortion 
and/or reduced signal attenuation, and may be interpreted 
more accurately by a signal recipient. 

The invention can also be applied to provide an audible 
signal representing P dimensions (P>2), formed or converted 
from a two-dimensional visual image region. The audible 
signal may, for example, provide depth clues, clues about a 
dominant hue or color or brightness, if any, and clues about 
the maximum fineness of detail associated with the image 
region, in addition to normal two-dimensional information. 50 

Consider a visual image region, such as a limited region of 
the image, and let p(t) represent an image region parameter 
that changes with time. The parameter may change continu- 
ously, or even differentiably, but in other more general situa- 55 
tions p(t) may also change by a discrete amount at each of a 
sequence of spaced apart times as {p(t„)}„, where 

p(t w )*p(t w+ i), as discussed in the preceding. Assuming that 
P(U+P(t o >0 for n=l, 2, . . . one can form a normalized 
parameter 


?(y = {p(o-p(^o)} / h'(^)+ J p( ? o)}. 

(15 A) 

P(tJp(t 0 )={\-q{Q}I{\+q(t n )}, 

(13B) 


which represents a difference or a ratio of a subsequent 65 
parameter value, relative to an initial or preceding parameter 
value. This difference or ratio can be represented audibly by 
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a baseline signal amplitude difference (b(t„)-b(t 0 )), ampli- 
tude ratio (b(t w )/b(t 0 )), envelope frequency difference (f M -f 0 ), 
or envelope frequency ratio (f„/f 0 ), among other combina- 
tions. 

As an example of application of the VISOR system, con- 
sider a battlefield situation in which one or more combatants, 
or one or more equipment items, are exposed to artillery 
shells or other projectiles, as illustrated in FIG. 6 and dis- 
cussed in more detail in Appendices A, B and C. 

In a first version, each combatant wears or carries a location 
determination (“LD”) system, such as GPS, and is aware of 
the combatant’s present location coordinates within an accu- 
racy of a few meters. A simple differential equation for a 
projectile ballistic trajectory is posited, and two or more 
observations, spaced apart in time, of the projectile location 
from each of two observers allows estimation of the relevant 
shell trajectory parameters, including projectile launch point, 
projectile impact point and time, and projectile explosive 
load, from which projectile injury and/or projectile lethality 
regions can be estimated. Where the combatants are or may be 
within the disability or injury or lethality region for the pro- 
jectile, the combatants can be notified collectively of this 
development by use of an audible (or visual) warning signal, 
such as a signal with monotonically decreasing (or increas- 
ing) frequency, with a final frequency value f(end) that is near 
to or below a frequency corresponding to disablement or 
injury or lethality. In this instance, each combatant receives a 
separate audible (or visual) warning signal with monotoni- 
cally varying frequency, having a final frequency f(end) that 
is specific for that combatant’s present location on the battle- 
field. That is, one combatant may be within an injury/lethality 
region, and another combatant may be outside this region, 
with a separate audible (or visual) warning signal for each. 

Where M (=3) observations of projectile location are pro- 
vided, projectile trajectory accuracy is enhanced by use of a 
statistically weighted average of trajectory location points. 
Distinction between trajectories of two or more projectiles 
that are present at substantially the same time is also avail- 
able. 

In a second version, the audible (or visual) warning signal 
has an undulatory signal frequency and/or an undulatory sig- 
nal intensity, which is different for a combatant location 
inside, as opposed to outside, a probable disability or injury or 
lethality region relative to the estimated impact site. 

In third and fourth versions, at least one (reference) com- 
batant, but less than all the combatants, wears or carries an LD 
system, and the injury or lethality region is estimated for the 
reference combatant. When the reference combatant is within 
the injury or lethality region, an appliance worn by the refer- 
ence combatant issues an audible (or visual) warning signal 
that is recognized by all nearby combatants. 

Consider a battlefield situation in which one or more com- 
batants, 61-/ (i=l, 2, . . . ) are exposed to artillery shells or 
other projectiles, as illustrated in FIG. 6. Each combatant, 
61-/ wears or carries an appliance 63-/ (i=l, 2, . . . ), including 
a receiver-processor for GPS signals and/or other location 
determination (“LD”) signals, received from LD transmitters 
65 -j (j=l, . . . , J; J=2) that are spaced apart from the com- 
batants. The appliance 65-/ associated with each combatant 
61-/ is aware of the appliance location coordinates (xi, yi, zi) 
to within an acceptably small inaccuracy. If differential GPS 
(“DGPS”) signals are used, the appliance location can be 
determined to within an inaccuracy of no more than one 
meter. 

A projectile 62 is launched from a launch site location LS, 
spaced apart from the combatants 61-/, roughly targeting the 
combatants and following a trajectory 67 that can be visually 
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or (preferably) electromagnetically observed and estimated. 

For example, a trajectory estimation system disclosed in 
Appendices A, B and C, or any other system with acceptable 
promptness of response, can be used for trajectory observa- 
tion and estimation. A trajectory observation and estimation 5 
system 66 observes and provides a prompt, accurate estima- 
tion of the proj ectile traj ectory 67, including but not limited to 
an estimate of the impact site location IS for the projectile 62. 
The location coordinates (x 67 , y 67 , z 67 ) for the projectile 
impact location 67 are promptly transmitted to each appliance 
63-/, which promptly computes the separation distance 

d(i]sev)={xi-x 61 ) 2 +(yi-y 61 ) 2 +{zi-z 61 ) 2 y 12 , ( 14 ) 

and generates an audible, time varying signal S a (t; i), illus- 15 
trated graphically in different versions in FIGS. 8A, 8B, 9A 
and 9B, that is communicated to the associated combatant 
61-/. Optionally, each combatant 61-1 receives a separately 
determined audible signal S^(t; i) that is chosen or customized 
for that combatant’s hearing system (including taking 20 
account of that combatant’s hearing acuity or audible signal 
sensitivity versus frequency). In one version, the audible sig- 
nal S f7 (t; i) begins at a relatively high, but audible perceptible 
frequency f(0), and quickly and monotonically decreases to 
an end frequency f(end) that is monotonically decreasing 25 
with decrease of separation distance d(i; sep) for the particu- 
lar combatant 61-/. Optionally, the audible signal S a (t; i) 
either terminates at the end frequency f(end) or continues at 
that end frequency. 

In a first version, the combatant 61-/ or the appliance 63-1 30 
compares the end frequency f(end) with a frequency f(cau- 
tion) (for which the combatant has been trained) to determine 
if the estimated impact location 67 of the projectile is close 
enough (within a distance d(E), depending upon the estimated 
explosive load E, illustrated in FIG. 7) to the combatant’s own 35 
location to possibly cause death or serious injury to an 
exposed combatant. If f (end) ^ f(cauti on) , the combatant 
quickly takes defensive maneuvers, such as reducing expo- 
sure to the projectile’s explosive force. If the estimated 
impact site location IS is further away and f(end)>f(caution), 40 
the combatant may elect to take no defensive maneuvers. 
Normally, a cautionary distance d(caution) varies inversely 
with an estimate of the explosive load E carried by the pro- 
jectile 62. 

.45 

In a second version, the appliance 63-/ numerically com- 
pares the estimated separation distance d(i; sep) with d(cau- 
tion), computed for the estimated explosive load of the pro- 
jectile 62, and communicates the result of this comparison 
audibly to the associated combatant 61-/, using an audible 5Q 
signal S^(t; i) with monotonically varying frequency f that 
decreases to f(end) (to be compared mentally with f(caution)) 
according to the separation distance d(i; sep). 

Alternatively, the audible signal frequency f(t) may 
increase monotonically as d(i; impact) decreases so that 55 
f(end)^f(caution) causes implementation of defensive 
maneuvers by the combatant 61-/. 

In the first version, each combatant 61-/ receives a sepa- 
rately determined audible signal S^(t; i). In the second ver- 
sion, applicable where the relative locations of a group of 60 
combatants is substantially unchanging (the combatants 
remain in place or move as a group), a single audible signal 
S^(t) can be provided, keyed to a separation distance d(ref; 
sep) of the estimated impact location from a reference com- 
batant (real or virtual) for the group. In this version, the 65 
frequency range f(end)^f^f(0) and the cautionary frequency 
f(caution) are preferably chosen to take account of the hearing 
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acuities of each member in the group of combatants. Different 
versions of this example are discussed in detail in Appendices 
A, B and C. 

As another example, consider a visual image region, a 
portion of a larger image, in which an obj ect of interest moves 
toward the viewer or away from the viewer at a substantial 
speed. Because of this movement, the apparent size (diameter 
viewed transverse to the direction of sight) of the object 
changes substantially with time. If the visual image is reduced 
in size and (re)defined so that this object is the dominant 
feature of the (resulting visual image, change of the diameter 
with time can be represented as an envelope frequency or 
baseline amplitude, for example, with envelope frequency or 
baseline amplitude changing in proportion to the increase (or 
decrease) with time of the diameter of the object. 

The system includes a “focus” mechanism (optional) that 
permits a visual image region (part of a larger image) to be 
discretely or continuously reduced or increased or otherwise 
adjusted in size (horizontally and vertically, independent of 
each other) to redefine, and focus on, a selected smaller visual 
image region, in order to more clearly display the image 
temporal changes that are of most importance. This adjust- 
ment in visual image region size can be implemented dis- 
cretely by drawing an initial quadrilateral or other polygon 
(rectangle, trapezoid, etc) as a border around the region to 
which the visual image is restricted. Optionally, the resulting 
visual image region, thus redefined, maintains its new shape, 
with an image diameter that increases or decreases according 
to the change in diameter that occurs as a result of definition 
of the selected smaller visual image region. 

Appendix A 

Example of Projectile Trajectory Estimation 

Where combatants are present on a battlefield and are 
exposed to artillery or armored vehicle fire, the injury and 
fatality count can be reduced if a probable impact point and a 
probable lethality radius for the shell or other proj ectile can be 
quickly estimated and communicated to the combatants. This 
requires some knowledge of the projectile trajectory, prefer- 
ably including probable launch point, probable explosive 
load and probable impact point, as well as knowledge of other 
flight variables. 

The approach provides a procedure for quickly estimating 
relevant projectile trajectory parameters and communicating 
this information in a format that allows substantially instan- 
taneous recognition of whether a combatant is inside or out- 
side a probable lethality or injury or disablement region for 
impact or explosion of the projectile. This information can be 
communicable to a group of adjacent combatants, to each 
combatant individually, and/or to users of equipment items on 
a battlefield. 

The approach is part of a system and method for (i) esti- 
mating relevant projectile trajectory parameters from two or 
more temporally spaced projectile observations from each of 
two or more spaced apart observers, (ii) estimating probable 
launch point coordinates, impact point coordinates, time of 
projectile impact and probable injury/1 ethality/disablement 
region (for each of two ore more combatants) from these 
parameters, (iii) use of (optional) subsequent observations of 
projectile locations to improve the accuracy of impact pre- 
diction and to distinguish between traj ectories of two or more 
projectiles that may be present at the same time. 

In a first version, each combatant wears or carries a location 
determination (“LD”) system, such as GPS, and is aware of 
the combatant’s present location coordinates within an accu- 
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racy of a few meters. A simple differential equation for a 
projectile ballistic trajectory is posited, and two or more 
observations, spaced apart in time, of the projectile location 
from each of two observers allows estimation of the relevant 
shell trajectory parameters, including projectile launch point, 
projectile impact point and time, and projectile explosive 
load, from which projectile injury and/or projectile lethality 
regions can be estimated. Where the combatants are or may be 
within the disability or injury or lethality region for the pro- 
jectile, the combatants can be notified or warned collectively 
of this development by use of an audible (or visual) warning 
signal, such as a signal with monotonically decreasing (or 
increasing) frequency, with a final frequency value f(end) that 
is near to or below a frequency corresponding to disability or 
injury or lethality. In this instance, each combatant receives a 
separate audible (or visual) warning signal with monotoni- 
cally varying frequency, having a final frequency f(end) that 
is specific for that combatant’s present location on the battle- 
field. That is, one combatant may be within an injury/lethality 
region, and another combatant may be outside this region, 
with a separate audible (or visual) warning signal for each. 

Where M (=3) observations of projectile location are pro- 
vided, projectile trajectory accuracy is enhanced by use of a 
statistically weighted average of trajectory location points. 
Distinction between trajectories of two or more projectiles 
that are present at substantially the same time is also avail- 
able. 

In a second version, the audible (or visual) warning signal 
has an undulatory signal frequency and/or an undulatory sig- 
nal intensity, which is different for a combatant location 
inside, as opposed to outside, a probable disability or injury or 
lethality region relative to the estimated impact site. 

In third and fourth versions, at least one (reference) com- 
batant, but less than all the combatants, wears or carries an LD 
system, and the injury or lethality region is estimated for the 
reference combatant. When the reference combatant is within 
the injury or lethality region, an appliance worn by the refer- 
ence combatant issues an audible (or visual) warning signal 
that is recognized by all nearby combatants. 

FIG. 6 illustrates a general environment, with two or more 
combatants, 61 -n (n=l, . . . , N; N^2), present and exposed to 
artillery shells or other projectiles 62 that are directed against 
the combatants. In one embodiment, each combatant 61 -n 
wears or carries a receiver-processor 63 -/?, including an 
antenna 64-/?, that receives location determination (“LD”) 
signals from Q spaced apart LD signal sources 65 -q (q= 
1, ***,Q;Q=3) an d estimates the present location coordi- 
nates, r„=(x w , y„, z„) of the combatant 61 -n or of the antenna 
64 -77. The receiver/processor 63-7? also receives trajectory 
parameter information (“TPI”) from one or more TPI sources 
66-p (p=l, . . . , P; Pi^l) that track and report TPI for one or 
more visible trajectories 67 corresponding to at least one 
projectile 62. Determination or estimation of trajectory infor- 
mation from two or more observers at two or more spaced 
apart observation times is presented in Appendix C. 

It is assumed here that the TPI signals for a given trajectory 
62 are received by a receiver-processor 63 -/?, including infor- 
mation that permits estimation of at least one of (i) launch site 
coordinates, r i =(x i , y L , z L ), (ii) impact site coordinates, r 7 = 
(x 7 , y 7 , z 7 ), and (iii) probable explosive load E of the projec- 
tile. The receiver-processor 63-7? computes a separation dis- 
tance 

(A-l) 
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and compares this separation distance with a probable impact 
effect distance, d(E)=d(E; injury or d(E; death), illustrated 
graphically in FIG. 7. 

If the Condition 

5 

d(n;sep)^d(E) (A-2) 

is satisfied so that injury or death from the projectile (which 
has not yet reached the impact site) is probable, the receiver- 
processor 63-7? generates a first audible (or visual) warning 
signal s a (t; n; 1) with a monotonically decreasing frequency, 
f=i z (t; n; deer) (FIG. 8A) or monotone increasing, f=f z (t; n; 
incr) (FIG. 8B). The distance d(E) may be different in differ- 
ent directions, measured from the estimated impact site, and 
5 may be different for each combatant, based on the protective 
gear worn by each combatant If the signal frequency f z (t; n; 
deer; 1) is monotone decreasing (FIG. 8A) and decreases 
below (preferably substantially below) a known cautionary 
frequency, f=f(caut), the corresponding combatant 61-t? is 
2Q made aware that he/ she is likely to suffer injury or death from 
explosion or impact of the incoming projectile, and the com- 
batant should quickly seek cover or protection, if possible. If 
the signal frequency, f=f z (t; n; incr; 1), is monotone increasing 
(FIG. 8B) and increases above (preferably substantially 
25 above) a known cautionary frequency, f=f(caut), the corre- 
sponding combatant 61-t? is made aware that he/ she is likely 
to suffer injury or death from explosion or impact of the 
incoming projectile, and the combatant should seek cover or 
protection, if possible. If d(n; sep)>d(E) so that injury or death 
30 from explosion of the incoming projectile is less likely or 
unlikely, the audible (or visual) signal frequency, f=f.(t; n; 
deer; 1) will stop decreasing at an end frequency, f(end)>f 
(caut), and the audible (or visual) signal frequency, f=f z (t; n; 
incr) will stop increasing at an end frequency, f(end)<f(caut), 
35 as illustrated in FIGS. 8A and 8B, respectively. 

In this first version, the location of each combatant 61-t? is 
known, to within an inaccuracy of a few meters or less, using 
an LD device that is part of the receiver-processor 63-/?, and a 
separation distance d(n; sep) is calculated for each combatant 
40 61-t?, preferably using information received at and/or com- 
puted by the corresponding receiver-processor 63 -n . The con- 
dition in Eq. (A-2) is tested separately for each combatant 
61-t? to determine if this condition is satisfied. For each com- 
batant 61-t? for which the condition (A-2) is satisfied, the 
45 receiver-processor 63-t? generates a first audible (or visual) 
warning signal s a (t; n; deer) in which the monotone decreas- 
ing frequency f z (t; n; deer) decreases to substantially below 
the cautionary frequency f(caut); or, alternatively, the mono- 
tone increasing frequency f z (t; n; incr) increases to substan- 
50 tially above the cautionary frequency f(caut). Where the con- 
dition (A-2) is not satisfied for a particular combatant 61-/?, 
the monotone decreasing frequency f z (t; n; deer) terminates or 
plateaus at an end frequency f(end)>f(caut); and the mono- 
tone increasing frequency f z (t; n; incr) terminates or plateaus 
55 at an end frequency f(end)<f(caut) . Optionally, the cautionary 
frequency f(caut) can be chosen separately for each combat- 
ant, to approximately coincide with a frequency of maximum 
audible (or visual) sensitivity for that combatant, to compen- 
sate for differences in frequency sensitivity between combat- 
60 ants - 

In a second version, where the condition (A-2) is satisfied 
for a combatant 61-/?, the system issues a second audible (or 
visual) warning signal s a (t; n; 2), which has (i) a rapidly 
varying, preferably undulatory, frequency f z (t; n; var) and/or 
65 (ii) a rapidly varying, preferably undulatory, signal intensity 
I z (t; n; var), as illustrated in FIGS. 9 A and 9B, to make the 
combatant aware of this condition. Where the condition (A-2) 


d(n;s^)={(x r x n ) 2 ^y r y n ) 2 +(zj-z n ) 2 } v \ 
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is not satisfied, the system issues a distinguishable audible (or 
visual) warning signal, for example, a constant frequency 
and/or constant intensity signal. In the first and second 
embodiments, the signal intensity of the output signal s z> (t) is 
preferably low enough, and/or the angular distribution of the 
signal intensity is sufficiently narrow, so that this warning 
signal is audible (or visible) only to an individual combatant 
61-77. 

In a third version, the receiver-processor 63-771 worn or 
carried by at least one combatant 6I-77I does not include an 
LD system so that this combatant and the associated receiver- 
processor 63-771 are not aware of the corresponding location 
coordinates of this combatant. In this situation, it is preferable 
that at least one reference receiver-processor 63-772 includes 
an LD system so that the corresponding reference combatant 
61-772 and the associated reference receiver-processor 63-772 
are aware of the corresponding location coordinates for this 
reference combatant. The receiver-processor 63-772 receives 
the TPI signals from the TPI source(s) 66 -p and determines 
the separation distance d(n; sep) for itself as in Eq. (A-l). If 
the condition (A-2) is satisfied, with n=n2, the receiver-pro- 
cessor 63-772 generates the audible (or visual) first warning 
signal s^(t; n; deer; 1), which is loud or intense enough to be 
recognized by any combatant within a selected distance D 
(e.g., D=50-100 meters) from the location of the reference 
combatant 63-772. As in the first embodiment, the first audible 
(or visual) warning signal s z (t; n; deer or incr) has a mono- 
tonically decreasing frequency, f=f 7z (t; n; deer) (FIG. 8A) ora 
monotone increasing frequency, f=f z (t; n; incr) (FIG. 8B), and 
this signal is recognized by all combatants within the distance 
D from the receiver-processor 63-772. The procedure for the 
remainder of the third embodiment proceeds as in the first 
embodiment. 

In a fourth version, at least one, but less than all, of the 
receiver-processors 63-772 includes an LD system, and if d(n; 
sep)^d(E), the system issues a second audible (or visual) 
warning signal s (7 (t; n; 2) with (i) a rapidly varying, preferably 
undulatory, frequency f^(t; n; var) and/or (ii) a rapidly vary- 
ing, preferably undulatory, signal intensity I z (t; n; var), as 
illustrated in FIGS. 9 A and 9B. The remainder of the fourth 
embodiment proceeds as in the second embodiment. 

In each of these versions, a receiver-processor 63-77 pro- 
vides a first audible (or visual) warning signal s a (t; n; deer or 
incr; 1) if the condition (A-2) is satisfied and provides a 
different, distinguishable warning signal if the condition 
(A-2) is not satisfied. Two or more distinguishable audible (or 
visual) warning signals, rather than text-based, symbol -based 
or color-based (visual) signals, are preferably used here, 
because an aural system is believed to be more flexible, to 
offer greater discrimination, and to offer greater range, than 
does a visual system. The launch site LS may be ground- 
based or may be part of an airborne or mobile vehicle. 

The invention can also be used to estimate an impact effect 
distance, d(E)=d(E; disablement), for disablement of an 
equipment item, such as a vehicle, a weapon or an observation 
instrument, that is located in a geographical region adjacent to 
the impact site, as graphically illustrated in FIG. 7. Alterna- 
tively, a visually perceptible or audibly perceptible warning 
signal may be issued to indicate that a separation distance 
d(sep) is no greater than an impact effect distance d(E) for (i) 
death of a combatant, (ii) serious injury to a combatant, or (iii) 
disablement of part or all of an equipment item, preferably 
with a perceptibly different signal being used for each of the 
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categories (i), (ii) and (iii). The different impact effect dis- 
tances for these three categories should be determined experi- 
mentally, at least in part. 

5 Appendix B 

Trajectory Equations 

10 Consider a projectile, such as an artillery shell, that is 
launched from a launch site with initially-unknown launch 
site coordinates (x i; y L , z L ), travels through an atmosphere, 
and is subject only to atmospheric and gravitational (g) 
forces, as illustrated in FIG. 6. The projectile 62 has a mass of 
1 5 m and an initial launch velocity vO immediately after launch, 
both initially unknown. A suitable second order differential 
equation describing ballistic motion of the projectile after 
launch from a launch site LS is 

2 q m(d 2 r/dt 2 )=F yv -kmg, (B-l) 

where F w is an estimated wind force vector (assumed con- 
stant), dependent upon the shape parameters and other rel- 
evant details of the projectile but not directly upon the mass 
25 m, and r(t) is location vector of the projectile at time t. 

Where the wind force is negligible or is ignored, a suitable 
solution of Eq. (B-l) is 

r{t)={{FJm)-kg}{t-tS) 2 l2+b{t-tti)+c, (B-2) 

30 

b=b(il, j2)={rh I . 2 )-K?n)}T^2-^i){( F y w )-^}( ? n +r z2- 

2t 0 ), (B-3) 

c=c(il,i2)={r(t n )+r(t i2 )+{(FJm)-k g}{(t a -t 0 ) 2 }}!2, (B-4) 

35 where r(t=t zl ) and r(t=t z2 ) are two observations of location 

coordinates for the projectile at distinct times t=t zl and t=t z2 
(1=1 , 2 , . . . ), and t 0 is a selected but arbitrary time value (e.g., 
t(;Hhi+t z -2)/2). Where the observation time values t zl and t z2 
are known, the solution r(t) can be extended backward and 
40 forward in time to estimate a launch time, t=t 7 , a correspond- 
ing launch site LS, an impact time, t=t 7 , and a corresponding 
impact site IS for which 

r (h)eSl, (B-5a) 

45 

r(t 7 )eS2, (B-5b) 

where S 7 is a known launch surface and S 7 is a known impact 
surface (e.g., planar or spheroidal). The launch time, t=t z , the 
50 impact time, t=t 7 , the launch site coordinates (x 7 , y L , z 7 ), and 
the impact site coordinates (x 7 , y 7 , z 7 ), are estimated from Eqs. 
(B-2), (B-3) and (B-4). 

Two or more projectile observations (at times t=t z l and t=t z2 
with t zl <t z2 ) from each of two or more spaced apart observers 
55 are used to provide trajectory vector values r(t zl ) and r(t z2 ). 
Where projectile observations are available at J>2 distinct 
times, t=yj=l, . . . , JS3), witht, 1 <t, 2 < . . . t, y , one can obtain 
a potentially more accurate estimation of the trajectory by 
replacing the trajectory parameters b(il,i2) and c(il,i2) with 
60 filtered or statistically weighted parameter values, b" and c", 
respectively, defined by 


J J (B - 6 ) 

^=Z Z h{ij, ij')b(ij, if), 

J= l f=j + 1 
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-continued 


I h(ij , if)c(ij, if) 
J= l J'=J+ 1 


where c is a velocity of signal propagation in the ambient 
medium. 

The line of sight segment L m from Om to P is described 
parametrically as follows for each observation point: 

x=d cos Q m cos ty m +x m , (C-2a) 


where h(ij,ij') are normalized, non-negative filter weights sat- 
isfying 


y=d cos Q m sin § m +y m 


z=d sin Q m +z m 




mm = i 


The projectile launch velocity 


(dr l dt) t=tL = v’o = [F w /m - kg }(t L -tO) + b, 


where (0,(|)) are spherical coordinates, referenced to the same 
(B-8) coordinate system, and d is the distance along the line of sight 
L m . From the Eqs. (C-2), one recovers consistency relations, 

1 5 (x-x m ) 2 +(y-y m ) 2 +(z-z m ) 2 =d 2 { cos 2 d m COS 2 (p m +COS 2 d m 

sin 2 (() w +sin 2 0 w }=d 2 (C-3) 

Equations (C-2a) and (C-2b) can be re-expressed as 

1 - 10 ) 

(x-jt^/jcos 0 m cos <|> m } = Cv-> ; m )/{cos 0 m sin (j 
20 zj/{sindj=d m . (C-4) 


determined immediately after launch, may be computed and 
used to estimate the proj ectile explosive load E, relying in part 
upon a database of launch velocity v 0 for each of the different 
projectiles in the adversary’s arsenal. The projectile explosive 25 
load E plus a reference curve or database of separation dis- 
tance d(E) for serious injury or lethality (FIGS. 8A and 8B) is 
used to estimate whether a combatant is within a probable 
injury or lethality region for the estimated impact site. 

Where the value of F w is known (e.g., from local wind 30 
observations), the projectile mass m can be estimated. By 
consulting an appropriate database of the adversary’s projec- 
tiles and comparing the projectile launch velocity, determined 
immediately after launch, plus the projectile mass m, an esti- 
mate of projectile explosive load E can be made. Using the 35 
information contained in FIG. 7 for the particular projectile 
used, an impact effect distance d(E) corresponding to serious 
injury or lethality or equipment disablement can be estimated 
and used in the preceding development in connection with Eq. 
(A-2) to determine which audible (or visual) warning signal 40 
,«(t) should be provided for a particular combatant, or for all 
combatants, in a given region. Two or more different warning 
signals may be provided, corresponding to different dangers 
from projectile impact. 

45 

Appendix C 
Trajectory Observations 

FIG. 10 illustrates a projectile that is observed at substan- 50 
tially the same time at each of two (or more) known observa- 
tion locations, Ol and 02, having the respective location 
coordinates (x l5 y l5 z x ) and (x 2 , y 2 , z 2 ). Each of these observ- 
ers observes a proj ectile P with unknown location coordinates 
(x p , y p , z p ) at substantially the same time, through measure- 55 
ments of a separation distance d m (m=l, 2) and spherical 
coordinates ((|) m ,0 m ), measured relative to a line of sight seg- 
ment L m from Om to P. The observer locations, Ol and 02 
and present projectile location P have coordinates referenced 
to an arbitrary but fixed coordinate system, including a plane 60 
II determined by line segments O-Ol and 0-02. 

A directly measured separation distance d m can be deter- 
mined by measuring a round trip return time of a radar or 
other electromagnetic, electronic or acoustic signal, issued at 
Om and received as a reflected return signal a time At m later, 65 


The location (Xp, y r , z P ) is the unique intersection of the 
line of sight segments and L 2 . From Eqs. (C-2a) and (C-2b) 

one infers that 


d\ = {det(M)} 1 -cos02sin02 cos02cos$>2 x 2 ~ x i 
d2 -cosflisin^i cos^cos^i yi—y\ 


det(A/)=cos r\ 2 cos r\ 2 sin^-^) 


x P =x l +d 1 cos 0 X cos 0 : cos § i =x 2 +d 2 cos 0 2 cos <|) 2 , 


yp = yi+d± cos 0 X sin <t> i = >' 2 +^2 cos ^2 si n ^2? 


z P =z l +d l sin d l =z 2 +d 2 \ 


from which the present location coordinates (Xp, y r , z P ) can 
be estimated. 

FIG. 11 illustrates a situation in which two (or more) dis- 
tinct projectiles, 62-1 and 62-2, following different trajecto- 
ries, 67-1 and 67-2, are present at the same time so that 
trajectory confusion is possible. For convenience, it is 
assumed that two observation sites, numbers m and m', are 
paired with each other for estimation of the present values of 
the projectile coordinates (Xp, y P , z p ) and that trans m ission of 
the interrogation signals from each of these two paired sites 
are coordinated, preferably occurring at substantially the 
same time. 

As a first approach to suppressing the possibility of confu- 
sion, each (first) observation site may use a different interro- 
gation frequency fl(interr) and I2(interr), such as a distin- 
guishable radar return frequency, and each observation site 
(m=l, 2) filters the return signal(s), discussed in the preced- 
ing, to identify its own return signal and to determine the 
observation site-projectile separation distance d QmP for itself. 
Where the first observation site (m) is also aware of the 
interrogation frequency f m ,(interr) used by the second obser- 
vation site (number m'^rn) to cooperatively determine the 
present location coordinates (xp, y P , z P ) of the projectile, as 
discussed in the preceding, the elapsed time At m , and separa- 
tion distance d 0m , P associated with a return signal for the 
second site can be estimated at the first site as 

At m .(est)=At m +(t m .-t m ), (C-10) 


d m (meas)=c(At m )/2(m=l ,2) 


(C-l) 


d m -(est)=c(At m -)/2 , 


(C-ll) 
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where t m and \ m > are the measured absolute times for receipt, 
at the first site, of the return signals with the associated return 
frequencies fl(interr) and f2(interr), respectively. The esti- 
mated separation distances, d m (est) and the directly measured 
separation distance d OT (meas), for the two paired observation 
sites can be compared with each other (for m=l and, sepa- 
rately, for m=2) to confirm or refute the hypothesis that each 
of the two paired sites is observing the same projectile. If each 
of the two paired sites is observing a different projectile, at 
least one of the two pairs of compared separation times, 
At m (est) and At m (meas), should be in substantial disagree- 
ment with the other corresponding separation time. 

This approach relies upon (1) filtering the return signals 
received in order to distinguish between the return signal 
frequencies fl(interr), f2(interr) and any other return signal 
frequency (2) use of separate return signal gating, at the first 
site and/or at the second site, of the return signals for the first 
and second sites. 

In a second approach, the projectile coordinates (x P , y r , 
2 p)t=a are determined from observations at times t=t z (i=l, 2, 
3, 4) with t 1 <t 2 «t 3 <t 4 . The projectile trajectory parameters 
for t^! and t=t 2 are compared with the corresponding pro- 
jectile trajectory parameters for t z =t 3 and t=t 4 . If the corre- 
sponding parameters are sufficiently close to each other, this 
tends to confirm the hypothesis that each of the two paired 
sites is observing the same projectile. If the two sets of cor- 
responding trajectory parameters are markedly different from 
each other, this indicates that each of the first and second 
observation sites is likely observing a different projectile. 

What is claimed is: 

1. A method for enhancing or extending a vision system of 
a human being, the method comprising providing a computer 
that is programmed: 

to represent at least one selected visual image region, hav- 
ing at least one color or hue associated with the region, 
in terms of at least one visual image parameter for the 
region, the at least one visual image parameter including 
at least one of: vertical coordinate range of the region; 
horizontal coordinate range of the region; center coor- 
dinates for the region; region brightness; dominant hue 
range or wavelength range of the region; change in a 
parameter value that characterizes the visual image, with 
respect to a reference parameter value; and time rate of 
change TRC in a parameter value that helps characterize 
the visual image region; 

to associate each of the visual image parameters with at 
least one of a first set of Ml audio signal attributes 
(Mli^l), the first set of attributes being drawn from 
signal carrier frequency, signal envelope frequency, car- 
rier signal-envelope signal phase difference at a selected 
time, baseline amplitude, envelope signal amplitude 
relative to baseline amplitude, and signal time duration, 
and with a second set of at least one of M2 audio signal 
attributes (M2i^l), the second set of attributes being 
drawn from time rate of change of envelope frequency 
and time-rate-of-change of baseline amplitude; 

to allow the selected visual image region to be adjusted in 
at least one of a horizontal dimension and a vertical 
dimension so that the visual image region primarily 
includes a selected object whose apparent diameter is 
increasing with time or is decreasing with time; and 

to present the M1+M2 audio signal attributes sequentially 
or simultaneously for the at least one selected region. 

2. The method of claim 1, wherein said computer is further 
programmed to provide a presentation format in which at 
least one of said signal carrier frequency, said signal envelope 
frequency, said signal carrier- signal envelope phase differ- 
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ence, said baseline amplitude, said time-rate-of-change of 
said baseline amplitude, said envelope signal amplitude rela- 
tive to said baseline 

amplitude, and said signal time duration changes mono- 
5 tonically with at least one of a coordinate representing 
said vertical location of said visual image region, a coor- 
dinate representing said horizontal location of said cor- 
responding visual image region, one or more coordi- 
nates representing a range of horizontal coordinates of 
10 said corresponding visual image region, one or more 
coordinates representing a range of vertical coordinates 
of said corresponding visual image region, a coordinate 
representing brightness of said corresponding visual 
image component, and a coordinate representing said 
15 dominant wavelength or dominant hue of said corre- 
sponding visual image region. 

3 . The method of claim 2, wherein said computer is further 
programmed to choose said monotonic change to be at least 
one of (a) a logarithmic change, (b) a power law change and 

20 (c) a linear change. 

4. The method of claim 2, wherein said computer is further 
programmed to choose said monotonic change to be a mono- 
tonically increasing change. 

5. The method of claim 2, wherein said computer is further 
25 programmed to choose said monotonic change to be a mono- 

tonically decreasing change. 

6. The method of claim 1, wherein said computer is further 
programmed: 

to determine said dominant hue or wavelength by a process 
30 comprising determining at least one selected wave- 
length range, drawn from wavelength ranges in ultravio- 
let, visible, near infrared and mid-infrared wavelengths, 
that contains said dominant hue or wavelength for said 
visual image component; and 

35 to associate said at least one of said audio signal attributes 
with the selected wavelength range. 

7. The method of claim 1, further comprising providing 
said at least one of said four audio signal attributes for a 
selected recipient whose sight is impaired. 

40 8. The method of claim 1, further comprising providing at 

least one of said signal carrier frequency and said signal 
envelope frequency in a frequency range determined by hear- 
ing acuity of a selected recipient. 

9 . The method of claim 1, further comprising choosing said 
45 at least one visual image component to have at least one 

selected wavelength that is part of at least one of an ultraviolet 
spectrum, a near-infrared spectrum and a mid-infrared spec- 
trum. 

10. The method of claim 1, further comprising represent- 
50 ing, to a selected recipient having an auditory communication 

system, said at least one of said audio signal attributes with at 
least one frequency to which the selected recipient’ s auditory 
communication system is sensitive. 

11. The method of claim 1, further comprising providing 
55 said visual image component for said region that cannot be 

accurately perceived visually because of at least one of signal 
interference, signal distortion and signal attenuation by an 
ambient environment. 

12. The method of claim 11, wherein said ambient envi- 
60 ronment includes at least one of rain, snow, ice, hail, sleet, 

fog, condensation, lightning, and transition between daylight 
and nighttime. 

13. The method of claim 1, further comprising choosing 
said at least one of said audio signal attributes so that said at 

65 least one audio signal attribute can be transmitted through an 
environment with at least one of reduced signal interference, 
reduced signal distortion and reduced signal attenuation. 
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14. The method of claim 1, further comprising analyzing 
said change in said parameter that characterizes said visual 
image. 

15. The method of claim 1, further comprising: 

identifying at least one of said time rate of change TRC of 5 

said parameter value that helps characterize said region; 
and 

representing said time rate of change TRC of said param- 
eter value as at least one of: (i) a continuous change with 
time, (ii) a discrete change at two or more spaced apart to 
times, and (iii) a discrete change, only when a magnitude 
of a difference between a first value of said parameter 
and a second value of said parameter is at least equal to 
a selected threshold magnitude. 

16 . A system for enhancing or extending a vision system of 15 
a human being, the system comprising providing a computer 
that is programmed: 

to represent at least one selected visual image region, hav- 
ing at least one color or hue associated with the region, 
in terms of at least one visual image parameter for the 20 
region, the at least one visual image parameter including 
at least one of: vertical coordinate range of the region; 
horizontal coordinate range of the region; center coor- 
dinates for the region; region brightness; dominant hue 
range or wavelength range of the region; change in a 25 
parameter value that characterizes the visual image, with 
respect to a reference parameter value; and time rate of 
change TRC in a parameter value that helps characterize 
the visual image region; 

to associate each of the visual image parameters with at 30 
least one of a first set of Ml audio signal attributes 
(MliM), the first set of attributes being drawn from 
signal carrier frequency, signal envelope frequency, car- 
rier signal-envelope signal phase difference at a selected 
time, baseline amplitude, envelope signal amplitude 35 
relative to baseline amplitude, and signal time duration, 
and with a second set of at least one of M2 audio signal 
attributes (M2iM), the second set of attributes being 
drawn from time rate of change of envelope frequency 
and time-rate-of-change of baseline amplitude; and 40 

to analyze an object, within the visual image region, that is 
moving toward a viewer or away from a viewer of the 
visual image region so that a lateral diameter of the 
object is increasing or is decreasing, respectively, and to 
transmit at least one of said M1+M2 audio signal 45 
attributes so that at least one of a signal frequency and a 
signal amplitude is increasing or decreasing in time at a 
rate corresponding to the increase or decrease in the 
lateral diameter with time; and 

to present the M1+M2 audio signal attributes sequentially 50 
or simultaneously for the at least one selected region. 

17. The system of claim 16, wherein said computer is 
further programmed to provide a presentation format in 
which at least one of said signal carrier frequency, said signal 
envelope frequency, said signal carrier- signal envelope phase 55 
difference, said baseline amplitude, said time-rate-of-change 
of said baseline amplitude, said envelope signal amplitude 
relative to said baseline amplitude, and said signal time dura- 
tion changes monotonically with at least one of a coordinate 
representing said vertical location of said visual image 60 
region, a coordinate representing said horizontal location of 
said corresponding visual image region, one or more coordi- 
nates representing a range of horizontal coordinates of said 
corresponding visual image region, one or more coordinates 
representing a range of vertical coordinates of said corre- 65 
sponding visual image region, a coordinate representing 
brightness of said corresponding visual image component, 
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and a coordinate representing said dominant wavelength or 
dominant hue of said corresponding visual image region. 

18. The system of claim 17, wherein said computer is 
further programmed to choose said monotonic change to be at 
least one of (a) a logarithmic change, (b) a power law change 
and (c) a linear change. 

19. The system of claim 17, wherein said computer is 
further programmed to choose said monotonic change to be a 
monotonically increasing change. 

20. The system of claim 17, wherein said computer is 
further programmed to choose said monotonic change to be a 
monotonically decreasing change. 

21. The system of claim 16, wherein said computer is 
further programmed: 

to determine said dominant hue or wavelength by a process 
comprising determining at least one selected wave- 
length range, drawn from wavelength ranges in ultravio- 
let, visible, near infrared and mid-infrared wavelengths, 
that contains said dominant hue or wavelength for said 
visual image component; and 

to associate said at least one of said audio signal attributes 
with the selected wavelength range. 

22. The system of claim 1 6, wherein said at least one of said 
four audio signal attributes is provided for a selected recipient 
whose sight is impaired. 

23. The system of claim 16, wherein at least one of said 
signal carrier frequency and said signal envelope frequency is 
provided in a frequency range determined by hearing acuity 
of a selected recipient. 

24. The system of claim 16, wherein said at least one visual 
image component has at least one selected wavelength that is 
part of at least one of an ultraviolet spectrum, a near- infrared 
spectrum and a mid-infrared spectrum. 

25. The system of claim 16, further comprising an auditory 
communication system that represents, to a selected recipi- 
ent, said at least one of said audio signal attributes with at least 
one frequency to which the selected recipient’ s auditory com- 
munication system is sensitive. 

26. The system of claim 16, wherein said visual image 
component for said region cannot be accurately perceived 
visually because of at least one of signal interference, signal 
distortion and signal attenuation by an ambient environment. 

27. The system of claim 26, wherein said ambient environ- 
ment includes at least one of rain, snow, ice, hail, sleet, fog, 
condensation, lightning, and transition between daylight and 
nighttime. 

28. The system of claim 1 6, wherein said at least one of said 
audio signal attributes is chosen so that said at least one audio 
signal attribute can be transmitted through an environment 
with at least one of reduced signal interference, reduced sig- 
nal distortion and reduced signal attenuation. 

29. The system of claim 16, wherein said computer is 
further programmed to analyze said change in said parameter 
that characterizes said visual image. 

30. The system of claim 16, wherein said computer is 
further programmed: 

to identify at least one of said time rate of change TRC of 
said parameter value that helps characterize said region; 
and 

to represent said time rate of change TRC of said parameter 
value as at least one of: (i) a continuous change with 
time, (ii) a discrete change at two or more spaced apart 
times, and (iii) a discrete change, only when a magnitude 
of a difference between a first value of said parameter 
and a second value of said parameter is at least equal to 
a selected threshold magnitude. 
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31 . A method for enhancing or extending a vision system of 
a human being, the method comprising providing a computer 
that is programmed: 

to represent at least one selected visual image region, hav- 
ing at least one color or hue associated with the region, 5 
in terms of at least one visual image parameter for the 
region, the at least one visual image parameter including 
at least one of: vertical coordinate range of the region; 
horizontal coordinate range of the region; center coor- 
dinates for the region; region brightness; dominant hue to 
range or wavelength range of the region; change in a 
parameter value that characterizes the visual image, with 
respect to a reference parameter value; and time rate of 
change TRC in a parameter value that helps characterize 
the visual image region; 15 

to associate each of the visual image parameters with at 
least one of a first set of Ml audio signal attributes 
(Mli^l), the first set of attributes being drawn from 
signal carrier frequency, signal envelope frequency, car- 
rier signal-envelope signal phase difference at a selected 20 
time, baseline amplitude, envelope signal amplitude 
relative to baseline amplitude, and signal time duration, 
and with a second set of at least one of M2 audio signal 
attributes (M2i=l), the second set of attributes being 
drawn from time rate of change of envelope frequency 25 
and time-rate-of-change of baseline amplitude; 
to analyze an object, within the visual image region, that is 
moving toward a viewer or away from a viewer of the 
visual image region so that a lateral diameter of the 
obj ect is increasing or is decreasing, respectively, and to 30 
transmit at least one of said M1+M2 audio signal 
attributes so that at least one of a signal frequency and a 
signal amplitude is increasing or decreasing in time at a 
rate corresponding to the increase or decrease in the 
lateral diameter with time; and 35 

to present the M1+M2 audio signal attributes sequentially 
or simultaneously for the at least one selected region. 

32. A system for enhancing or extending a vision system of 

a human being, the system comprising providing a computer 
that is programmed: 40 

to represent at least one selected visual image region, hav- 
ing at least one color or hue associated with the region, 
in terms of at least one visual image parameter for the 
region, the at least one visual image parameter including 
at least one of: vertical coordinate range of the region; 45 
horizontal coordinate range of the region; center coor- 
dinates for the region; region brightness; dominant hue 
range or wavelength range of the region; change in a 
parameter value that characterizes the visual image, with 
respect to a reference parameter value; and time rate of 50 
change TRC in a parameter value that helps characterize 
the visual image region; 

to associate each of the visual image parameters with at 
least one of a first set of Ml audio signal attributes 
(Mli^l), the first set of attributes being drawn from 55 
signal carrier frequency, signal envelope frequency, car- 
rier signal-envelope signal phase difference at a selected 
time, baseline amplitude, envelope signal amplitude 
relative to baseline amplitude, and signal time duration, 
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and with a second set of at least one of M2 audio signal 
attributes (M2^1), the second set of attributes being 
drawn from time rate of change of envelope frequency 
and time-rate-of-change of baseline amplitude; 
to allow the selected visual image region to be reduced in at 
least one of a horizontal dimension and a vertical dimen- 
sion so that the visual image region primarily includes a 
selected object whose apparent diameter is increasing 
with time or is decreasing with time; and 
to present the M1+M2 audio signal attributes sequentially 
or simultaneously for the at least one selected region. 

33 . A method for enhancing or extending a vision system of 
a human being, the method comprising providing a computer 
that is programmed: 

to represent at least one selected visual image region, hav- 
ing at least one color or hue associated with the region, 
in terms of at least one visual image parameter for the 
region, the at least one visual image parameter including 
at least one of: vertical coordinate range of the region; 
horizontal coordinate range, of the region; center coor- 
dinates for the region; region brightness; dominant hue 
range or wavelength range of the region; change in a 
parameter value that characterizes the visual image, with 
respect to a reference parameter value; and time rate of 
change TRC in a parameter value that helps characterize 
the visual image region; 

to associate each of the visual image parameters with at 
least one of a first set of Ml audio signal attributes 
(Mli^l), the first set of attributes being drawn from 
signal carrier frequency, signal envelope frequency, car- 
rier signal -envelope signal phase difference at a selected 
time, baseline amplitude, envelope signal amplitude 
relative to baseline amplitude, and signal time duration, 
and with a second set of at least one of M2 audio signal 
attributes (M2i=l), the second set of attributes being 
drawn from time rate of change of envelope frequency 
and time-rate-of-change of baseline amplitude; 
to present the M1+M2 audio signal attributes sequentially 
or simultaneously for the at least one selected region as 
an audibly perceptible signal (APS) that is representable 
as an equation 

S a (t)=b(t)+a(t ) sin {.f e (r)M-<I> J sin {f c t+® c } 

where b(t) is a baseline amplitude for the APS, expressed as a 
function of time t and having a time rate of change db/dt 
associated with the baseline amplitude, a(t) is an envelope 
signal amplitude, f e (t) is a signal envelope frequency, which 
may vary with the time t, (|) e is a signal envelope phase, f c is a 
carrier signal frequency, and <D C a carrier signal phase; and 
to analyze an object, within the visual image region, that is 
moving toward a viewer or away from a viewer of the 
visual image region so that a lateral diameter of the 
object is increasing or is decreasing, respectively, and to 
transmit at least one of said M1+M2 audio signal 
attributes so that at least one of a signal frequency, jJX) 
or f c , and a signal amplitude, a(t) or b(t), is increasing or 
decreasing in time at a rate corresponding to the increase 
or decrease in diameter with time. 



